Multispectral sensing system and method

ABSTRACT

A multispectral sensing device is disclosed. The sensing device may comprise an array of pixel units. Each of the pixel units may comprise four pixels in a two by two configuration. Each of the pixels may comprise a plurality of sub-pixels. Each of the pixel units may include at least one pixel that includes at least two sub-pixels configured to detect light of different wavelengths.

TECHNICAL FIELD

The present application relates to the technical field of image sensing,and more particularly, to multispectral sensing.

BACKGROUND

Multispectral sensing collects and processes information from across theelectromagnetic spectrum. Multispectral images are used in the fieldssuch as remote sensing, agriculture, product quality assessment,security, and surveillance. Conventional multispectral sensing systemscapture different spectral channels of the scene, by using either afilter wheel or a beam splitter combined with multiple detectors. Suchsystems are inadequate for producing high quality images and may requirecomplicated implementation steps.

SUMMARY

One aspect of the present disclosure is directed to a multispectralsensing device. The sensing device may comprise an array of pixel units.Each of the pixel units may comprise four pixels in a two by twoconfiguration. Each of the pixels may comprise a plurality ofsub-pixels, and each of the pixel units may include at least one pixelthat includes at least two sub-pixels configured to detect light ofdifferent wavelengths.

Another aspect of the present disclosure is directed to a multispectralsensing device. The sensing device may comprise an array of pixels. Eachof the pixels may comprise a plurality of sub-pixels, and at least twoof the sub-pixels in one pixel of the array may be configured to detectlight of different wavelengths.

Another aspect of the present disclosure is directed to a multispectralsensing system. The system may comprise a Bayer-pattern array of pixelsand a processing circuit. Each pixel may comprise a plurality ofsub-pixels. The processing circuit may be coupled to each of thesub-pixels and configured to tune a detection wavelength band of thesub-pixel, causing the sub-pixels to be switchable between detectinglight of the same and different wavelength bands. Switching between thevarious modes can allow the sensing device to detect light of differentwavelengths/wavelength bands, thereby producing spectral information ofthe scene.

Additional features and advantages of the present disclosure will be setforth in part in the following detailed description, and in part will beobvious from the description, or may be learned by practice of thepresent disclosure. The features and advantages of the presentdisclosure will be realized and attained by means of the elements andcombinations particularly pointed out in the appended claims.

It is to be understood that the foregoing general description and thefollowing detailed description are exemplary and explanatory only, andare not restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which constitute a part of thisspecification, illustrate several embodiments and, together with thedescription, serve to explain the disclosed principles.

FIG. 1A is a graphical representation illustrating a Bayer-pattern unitof an image sensor in prior art from both top and side views.

FIG. 1B is a graphical representation illustrating a Bayer-pattern unitof a multispectral sensor from both top and side views, consistent withthe exemplary embodiments of the present disclosure.

FIG. 2 is a schematic circuit diagram illustrating a multispectralsensor system, consistent with the exemplary embodiments of the presentdisclosure.

FIGS. 3A and 3B are graphical representations illustrating filteringrange tuning in a multispectral sensor, consistent with the exemplaryembodiments of the present disclosure.

FIGS. 4A and 4B are graphical representations illustrating sensingreading tuning in a multispectral sensor, consistent with the exemplaryembodiments of the present disclosure.

DETAILED DESCRIPTION

Reference will now be made in detail to exemplary embodiments, examplesof which are illustrated in the accompanying drawings. The followingdescription refers to the accompanying drawings in which the samenumbers in different drawings represent the same or similar elementsunless otherwise represented. The implementations set forth in thefollowing description of exemplary embodiments consistent with thepresent disclosure do not represent all implementations. Instead, theyare merely examples of systems and methods consistent with aspectsrelated to the invention as recited in the appended claims.

Conventional approaches for capturing multispectral images are notideal. Most of such technologies cannot produce high quality images andrequire complicated implementation steps. For example, a wheel of colorfilters placed between an object and a regular detector may be used tocapture various color/wavelength bands. Since the wavelength bands arecaptured at different time stamps, the exposure time for the entireimage is limited by the speed of rotation of the wheel, which usuallytakes more time than common shutter operations. Such multi-exposureimages may appear more blurry than single-exposure images. Further, thefilter and the detector have to be synchronized precisely, because aslight mismatch may generate image defects. For another example, a beamsplitter may be used to split light from an object into various colorbands refracted to corresponding sensors. Similarly, this system mayrequire precise optical calibration before implementation. Further,separating the colors to different paths is likely to introducesignificant path length differences and cause errors in imaging.

A claimed solution rooted in semiconductor engineering overcomesproblems arising in the realm of electronic and optical technology. Invarious embodiments, a multispectral sensing device is disclosed. Thesensing device may comprise an array (e.g., a planar array) of pixelunits. Each of the pixel units may comprise four pixels (e.g., fourpixels arranged in a two by two configuration). In another word, thesensing device may comprise a plurality of pixels in a planar array. Forexample, the pixel array of the sensing device may be in a Bayer patternand referred to as a Bayer-pattern array. Accordingly, the pixel unitmay be referred to as a Bayer-pattern unit. Each of the pixels maycomprise a plurality of sub-pixels, and at least two of the sub-pixelsmay be configured to detect light of different wavelengths. For example,the plurality of sub-pixels may be four (or another number of)sub-pixels, such that each of the pixel units comprises sixteensub-pixels, and the sixteen sub-pixels may be configured to detect lightof different wavelengths or wavelength bands. In some embodiments, eachof the sub-pixels may comprise a μlens, an optical filter, and aphotodiode. The μlens may be disposed above the optical filter, thephotodiode may be disposed below the optical filter, the μlens may beconfigured to direct impinging light to pass through the optical filter,and the photodiode may be configured to detect the light passed throughthe optical filter. Thus, the sensing device can detect multispectrallight (e.g., sixteen different wavelength bands) without deployingmechanical filters, beam splitters, or other complicated opticalcomponents. Further, when using this sensing device, an image can beobtained by a single exposure of the device, such that the multispectralsensing can be achieved without sacrificing the image quality.

In some embodiments, the four sub-pixels are configured to detect lightof different wavelengths. For example, three of the four sub-pixels maybe configured to detect visible light, and the other sub-pixel of thefour sub-pixels may be configured to detect infrared (IR) light. Forexample, the three sub-pixels are configured to detect red, blue, andgreen light respectively, and the other sub-pixel is configured todetect near infrared (NIR) light.

In some embodiments, the wavelength sensing range of each sub-pixel istunable via a processing circuit. For example, in each sub-pixel, theoptical filter is coupled to the processing circuit. The processingcircuit may be configured to tune a filtering range of the opticalfilter by controlling one or more properties of the optical filter.Thus, the wavelength sensing range of each sub-pixel can be tuned bytuning the filtering range of the corresponding optical filter.

In some embodiments, the processing circuit is coupled to the photodiodeof each sub-pixel and configured to tune the reading of themultispectral sensing device. The processing circuit may be configuredto receive readings of the photodiodes and output the reading of themultispectral sensing device based on one or more of the receivedreadings. In one example, the processing circuit is configured to obtaina total reading of the readings of all photodiode readings in a pixel,and output the total reading as the pixel's reading. In another example,the processing circuit is configured to obtain one photodiode's readingfrom all photodiodes in a pixel, determine the pixel's reading based onthe obtained reading, and output the determined reading as the pixel'sreading.

In some embodiments, a multispectral sensing device is disclosed. Thesensing device may comprise an array of pixels. Each of the pixels maycomprise a plurality of sub-pixels, and at least two of the sub-pixelsmay be configured to detect light of different wavelengths or wavelengthbands. For example, the array may be a Bayer-pattern array, and theplurality of sub-pixels may be configured to detect light of awavelength or wavelength band different from each other and fromsub-pixels of any of eight closest neighboring pixels. For square pixelsarranged in the Bayer-pattern array (e.g., in a chess boardconfiguration), any non-border square pixel is surrounded by eightsimilar square pixels in corresponding relative positions: left, right,front, rear, front left, front right, rear left, and rear right.Similarly, the sensing device may further comprise a processing circuitcoupled to the photodiode as discussed above. Thus, the sensing devicecan detect multispectral light without deploying mechanical filters,beam splitters, or other complicated optical components. Further, whenusing this sensing device, an image can be obtained by a single exposureof the device, such that the multispectral sensing can be achievedwithout sacrificing the image quality.

In some embodiments, a multispectral sensing system is disclosed. Thesystem may comprise a Bayer-pattern array of pixels each comprising aplurality of sub-pixels. The system may further comprise a processingcircuit coupled to each of the sub-pixels and configured to tune adetection wavelength band of each sub-pixel, causing the sub-pixels tobe switchable between detecting light of the same and differentwavelength bands. In each Bayer-pattern unit, the pixels may comprisered, green, blue, and infrared pixels. Each of the sub-pixels maycomprise an optical filter. The detection wavelength band of thesub-pixel may be tunable through tuning a filtering range of the opticalfilter. Thus, each pixel can detect one wavelength/wavelength band, twowavelengths/wavelength bands, three wavelengths/wavelength bands, fourwavelengths/wavelength bands, etc.

In some embodiments, one or more of the μlens, filter, and photodiode ofeach sub-pixel may be wavelength-specific. For example, the μlens oroptical filter of the NIR sub-pixel may select NIR light or allow NIRlight to pass through. For another example, the photodiode of the NIRsub-pixel may detect NIR light. Thus, for each sub-pixel, it is notnecessary to require all of the μlens, filter, and photodiode to bewavelength-specific, as long as at least one of them is. Further, thewavelength sensing range of the sub-pixel can thus be tuned by tuning afiltering range of the corresponding optical filter.

The devices and systems disclosed above may apply to visible sensing,non-visible sensing, or a combination of both. For example, the pixelsof each pixel unit may comprise a red pixel, a blue pixel, a greenpixel, and an IR pixel. IR radiation spans wavelengths betweenapproximately 750 nanometers (nm) and 1 millimeter (mm). One commonlyused sub-division scheme of IR can include near IR (0.75-1.1 μm); mid-IR(1.1-30 μm); and far IR (30-1,000 μm). In this specification, someembodiments use NIR as an example. The optical filters disclosed hereinmay include, but are not limited to, visible light color filters (e.g.,red color filter, green color filter, etc.) and IR filters (e.g., NIRcolor filter, mid-IR color filter, etc.). For example, the opticalfilter can be a NIR filter that selects NIR light or allows NIR light topass through. Other filters, like IR filter, mid IR filter, or long IRfilter, can be used as the optical filter. In some other embodiments,filters that are substantially transparent to ultraviolet (UV) light maybe used for UV imaging. Corresponding photodiodes sensitive to UV lightmay be placed under the UV filters in these types of sensing devices.Similarly, the IR or RGB pixel (including its associated μlens, filter,and photodiode) may be replaced by another pixel of any wavelength toachieve multispectral sensing disclosed herein.

FIG. 1A is a graphical representation 110 illustrating a Bayer-patternunit of an image sensor in prior art from both top and side views. Themultispectral sensor may be a solid-state (e.g., CCD, CMOS) sensor. Suchsensor comprises pixels in a Bayer-pattern array. A typicalBayer-pattern unit is shown in FIG. 1A top view. From the top view, fourpixels of the Bayer-pattern unit are shown as four squares, associatedwith a green filter, a red filter, a blue filter, and a NIR filter. Eachpixel or filter has the identical dimension of width X. A typical valuefor width X is 3.0 μm. Each square comprises a circle representing amicro lens (μlens). Referring to FIG. 1A side view, for each pixel, theμlens is disposed on the associated optical filter (OF), which is on topof an associated photodiode (PD). The optical filter may select red,green, blue, or NIR light.

FIG. 1B is a graphical representation 120 illustrating a Bayer-patternunit of a multispectral sensor from both top and side views, consistentwith the exemplary embodiments of the present disclosure. Based on FIG.1B, a multispectral sensing device as disclosed may comprise an array ofsuch Bayer-pattern units.

The graphical representation 120 shows four pixels each of width X, thefour pixels forming the Bayer-pattern unit. In some embodiments, thepixels may be configured to detect red, green, blue, and NIR.Alternatively, each pixel may be configured to detect anotherwavelength. Each pixel may comprise multiple sub-pixels of width Y. Inthis figure, each pixel of width X comprises four sub-pixels of width Y.The width Y and width X can have various lengths. An exemplary lengthfor Y is 1.5 μm, and an exemplary length for X is 3.0 μm.

The Bayer-pattern described herein may include various modifications inaddition to those shown in FIGS. 1A and 1B. In some embodiments, theoptical filters, pixels, and sub-pixels in a Bayer-pattern unit mayinclude any color, such as RGBG (red, green, blue, green), CYGM (cyan,yellow, green, magenta), and RGBE (red, green, blue, emerald), etc. Thatis, the optical filters, pixels, and sub-pixels are not limited to red,green, and blue. Moreover, multiple pixels and corresponding filters ina Bayer-pattern unit may be oriented in various configurations, e.g.,rotated by 45 degrees. Further, each pixel can be divided into anynumber of sub-pixels of any shapes, and each sub-pixel can beindividually controlled in terms of the integration time, therebyachieving multiple integration times within the same pixel. Therefore,the disclosed systems, methods, and devices similarly apply to any ofthe modified Bayer-pattern sensor designs.

From the top view as shown, the Bayer-pattern unit may comprise fourpixels (e.g., one of the pixels being pixel 121). Each pixel maycomprise four sub-pixels (e.g., the pixel 121 comprises sub-pixels 122,123, 124, and 125). The four pixels may be configured to detect red,blue, green, and NIR light respectively. Here, the red, blue, green, andNIR light may each refer to a wavelength range, within which thecorresponding sub-pixels in each pixel can detect various wavelengthbands. For example, the sub-pixels in the green pixel may be configuredto detect dark green, green, light green, etc. Thus, for theBayer-pattern unit shown in this figure, up to sixteen differentwavelengths/wavelength bands can be detected simultaneously to achievemultispectral sensing. Each of the sub-pixels may be labeled by acorresponding wavelength (e.g., B1=420 nm, B2=450 nm, B3=480 nm, B4=500nm, G1=520 nm, G2=550 nm, G3=580 nm, G4=600 nm, R1=620 nm, R2=650 nm,R3=680 nm, R4=700 nm, N1=750 nm, N2=850 nm, N3=900 nm, N4=950 nm,wherein B refers to blue, G refers to green, R refers to red, and Nrefers to NIR) or wavelength band (e.g., B1=415-425 nm, B2=445-455 nm,B3=475-485 nm, B4=495-505 nm, G1=515-425 nm, G2=545-555 nm, G3=575-585nm, G4=595-605 nm, R1=615-625 nm, R2=645-655 nm, R3=675-685 nm,R4=695-705 nm, N1=745-755 nm, N2=845-855 nm, N3=895-905 nm, N4=945-955nm). In some embodiments, the sub-pixels do not have to be grouped bywavelength. That is, the sixteen sub-pixels in each Bayer-pattern unitmay be configured to detect any wavelength band, and their positionswith respect to each Bayer-pattern unit plane may be random.

From the side view as shown, each sub-pixel may include a photodiode, aμlens, and an optical filter (OF). In this side view, only one row ofthe sub-pixels is shown. Thus, only two sub-pixels are shown for eachpixel. The μlens is disposed above the optical filter, and thephotodiode is disposed below the optical filter. Each μlens may besubstantially aligned with its corresponding photodiode, such that mostof the photons impinging on the μlens can be directed to pass theoptical filter and detected by the photodiode. The photodiode generatescharges (e.g., electrons and holes) in response to the light exposure toperform the detection. Each photodiode may be coupled to a processingcircuit (e.g., via a column readout circuit described below withreference to FIG. 2). It is also possible that two or more photodiodesshare the same coupling channel to the processing circuit.

In some embodiments, each optical filter is coupled to the processingcircuit, and the sensing range of each sub-pixel is tunable via theprocessing circuit. For example, the optical filter may be a liquidcrystal tunable filter, an acousto-optic tunable filter, anelectromagnetic glass filter, etc. The processing circuit may beconfigured to control or actuate a current or voltage supplied to eachoptical filter to effectuate the filtering range tuning. Thus, thewavelength sensing range of each sub-pixel can be tuned by tuning thefiltering range of the corresponding optical filter from the processingcircuit. Alternatively, one or more the optical filters may benon-tunable and designed to allow light of a corresponding wavelength orwavelength band to pass.

In some embodiments, the processing circuit is coupled to the photodiodeof each sub-pixel and configured to tune the reading of themultispectral sensing device. The processing circuit may be configuredto receive readings of the photodiodes and output the reading of themultispectral sensing device based on one or more of the receivedreadings. In one example, the processing circuit is configured to obtaina total reading of all photodiode readings in each pixel, and output thetotal reading as the pixel's reading. There may be various methods toobtain the total reading. For example, electron signals generated at thesub-pixels from an exposure may be converted to voltage signals at anoutput node or at the corresponding pixel or sub-pixel. Then, theprocessing circuit may receive the voltage signals pixel by pixel (e.g.,via binning) or sub-pixel by sub-pixel (e.g., via individual photodiodereadout). With respect to the binning, the photodiodes in each pixel maybe binned together and read by the processing circuit. For example, FIG.1B shows two photodiodes in a pixel binned to a floating diffusion node130. Since FIG. 1B is only a side view and a pixel may comprise foursub-pixels arranged in a two by two configuration, four photodiodescorresponding to four sub-pixels in a pixel may be binned to a floatingdiffusion node. With respect to the individual photodiode readout, thephotodiodes in each pixel may be individually read by the processingcircuit, and the processing circuit may sum the photodiode signals foreach pixel. In another example, the processing circuit is configured toobtain the reading of one photodiode out of all photodiodes in eachpixel, determine the pixel's reading based on the obtained reading(e.g., by multiplying the obtained reading by the number of photodiodesin the pixel), and output the determined reading as the pixel's reading.Similarly, the processing circuit may obtain the reading(s) of anysubset of photodiode(s) out of all photodiodes in each pixel todetermine the pixel's reading. U.S. application Ser. No. 15/363,873,filed on Nov. 29, 2016, and titled “System and Method for High DynamicRange Image Sensing”, and U.S. application Ser. No. 15/415,376, filed onJan. 25, 2017, and titled “System and Method for Visible and InfraredHigh Dynamic Range Sensing” provide more details on the processingcircuit, binning circuits and methods, and methods on obtaining readingsfrom the pixels. Both of the applications are assigned to the sameassignee as the present application and both are incorporated herein byreference.

A person having ordinary skill in the art should be able to selectphotodiodes with particular materials sensitive to certain wavelengths.For example, photodiodes that are sensitive to visible light can beplaced under RGB filters. Photodiodes that are sensitive to infraredlight, or particularly to near infrared light, such as siliconphotodiodes (with broad detection spectrum and particular thickness) canbe placed under the NIR filter. The μlens and the optical filter mayhave a working range covering the detection range of the correspondingphotodiode, such that even if the detection range changes, thecorresponding μlens and optical filter do not have to change.

FIG. 2 is a schematic circuit diagram illustrating a multispectralsensing system 200, consistent with the exemplary embodiments of thepresent disclosure. The system 200 may comprise a pixel array 201, a rowcontrol circuit 202, a column readout circuit 203, and a processingcircuit 204. The pixel array 201 may comprise multiple pixels arrangedby row and column and associated circuits, e.g., pixels shown in FIG.1B. Each pixel row is coupled to row control circuit 202, and each pixelcolumn is coupled to the column readout circuit 203. The row controlcircuit 202 may control readout timing in each row, and the columnreadout circuit 203 may receive pixel signals from each column. Theprocessing circuit 204 may control the row control circuit 202 and thecolumn readout circuit 203 in terms of clock timing, signal processing(e.g. receiving readings from the photodiodes and processing thereadings), signal control (e.g., controlling the current and/or voltagesupplied to each optical filter), etc. The above circuit is merelyexemplary, and other possible variations of circuit control may beimplemented.

During one exemplary exposure of the photodiode sensor, the photodiodesin the pixel array 201 may convert impinging photons into charges (e.g.,electrons and/or holes). The charges are integrated (collected) incorresponding sensor sub-pixels or pixels. After the completion ofintegration cycle, the collected charges are converted into voltages.The voltages are supplied to the output terminals and coupled to columnreadout circuit 203. In CMOS image sensors, the charge to voltageconversion is accomplished directly in the pixels themselves. The analogpixel voltage is transferred to the output terminals through variouspixel addressing and scanning schemes controlled by the row controlcircuit 202. The analog signal can also be converted on-chip to adigital equivalent before reaching the chip output.

In some embodiments, the processing circuit 204 may be configured totune the detection wavelength band of each of the photodiodes, such thatthe photodiodes in each pixel may be configured to detect the samewavelength band, or two or more different wavelength bands as discussedbelow with reference to FIGS. 3A and 3B.

FIGS. 3A and 3B are graphical representations illustrating filteringrange tuning in a multispectral sensor, consistent with the exemplaryembodiments of the present disclosure. FIGS. 3A and 3B each show aBayer-pattern unit of sixteen sub-pixels in mode 310 and mode 320 with aprocessing circuit. Each of the sub-pixels is labelled to indicate itsdetection wavelength. For example, the top left sub-pixels in bothfigures are configured to detect light of wavelength G1. In FIG. 3A, allsixteen photodiodes are configured to detect different wavelengths(e.g., G1, G2, B1, B2, etc.). G1, G2, G3, and G4 may refer to fourdifferent wavelength substantially in the green wavelength range, butdifferent from each other. The mode 310 can be switched to the mode 320or a different configuration and vice versa. To switch to the mode 320,G3 and G4 in the mode 310 are tuned to G1, so the green pixel comprisestwo different detection wavelengths G1 and G2; B1 is tuned to B2, so theblue pixel comprises three different detection wavelengths B2, B3, andB4; R1, R2, R3, and R4 are unchanged, so the red pixel still comprisesfour different detection wavelength; N1, N2, N3, and N4 are all tuned toN, so the NIR pixel comprises one detection wavelength N. The wavelengthtuning may be achieved by the filtering range tuning described above,since the individual optical filter may be actuated by the processingcircuit in terms of its filtering range. Similarly, the mode 320 can beswitched back to the mode 310. Thus, the optical filters in each pixelmay be switchable between having the same filtering range and differentfiltering ranges, causing the sensing device to be switchable, forexample, between sensing more than four wavelength bands and fourwavelength bands. Switching between the various modes can allow thesensing device to detect light of different wavelengths/wavelengthbands, thereby producing spectral information of the scene.

The modes 310 and 320 are merely exemplary. The above exampleillustrates that each pixel can be tunable among detecting one, two, . .. , N wavelength ranges, where N is the number of sub-pixels of thepixel. By the same principle, each Bayer-pattern unit may be tunableamong from detecting one, two, . . . , M wavelength ranges, where M isthe number of sub-pixels of the Bayer-pattern unit.

FIGS. 4A and 4B are graphical representations illustrating sensingreading tuning in a multispectral sensor, consistent with the exemplaryembodiments of the present disclosure. FIGS. 4A and 4B each show aBayer-pattern unit of sixteen sub-pixels in mode 410 and mode 420 with aprocessing circuit. Similar to FIG. 3A, each of the sub-pixels in FIG.4A is labelled to indicate its detection wavelength.

In some embodiments, the processing circuit may be configured to receivereadings of the sixteen photodiodes and output the reading of theBayer-pattern unit based on one or more of the received readings. Forthe green pixel, the processing circuit may determine the green pixelreading based on the G1 reading alone. Thus, in mode 420, the processingcircuit may multiply G1 by 4 (the number of sub-pixels) to obtain 4G1 asthe output of the green pixel. For the blue pixel, the processingcircuit may replace B4 with B3, to obtain (B1+132+2B3) as the output ofthe blue pixel. For the red pixel, the processing circuit may replace R2with R1 and replace R3 with R4, to obtain 2(R1+R4) as the output of thered pixel. For the NIR pixel, the processing circuit may add up allsub-pixel readings (N1+N2+N3+N4) to obtain the output of the NIR pixel.The method applied to the NIR pixel may be known as binning, as thephotodiodes in this pixel are binned together to derive a total reading.The mode 410 and the mode 420 may be switchable back and forth by theprocessing circuit.

The reading methods for the pixels above are merely exemplary. Theprocessing circuit may determine the reading method for each pixel basedon various conditions, such as user settings, sensing conditions, pixelconditions, etc. It is possible to apply the binning method to allpixels in a multispectral sensing device, or determine one from allphotodiode readings in each pixel to obtain the pixel's reading. Thus,the sensing device can detect multispectral light without deployingmechanical filters, beam splitters, or other complicated opticalcomponents. Further, enabled by electronic signals for mode switching,the sensing device can flexibly detect a configurable number ofwavelengths/wavelength bands.

The specification has described methods, apparatus, and systems formultispectral sensing. As shown above, each sensing device or system cansimultaneously detect light of n different wavelengths, with n being anynatural number and as long as the fabrication permits. The illustratedsteps are set out to explain the exemplary embodiments shown, and itshould be anticipated that ongoing technological development will changethe manner in which particular functions are performed. Thus, theseexamples are presented herein for purposes of illustration, and notlimitation. For example, steps or processes disclosed herein are notlimited to being performed in the order described, but may be performedin any order, and some steps may be omitted, consistent with thedisclosed embodiments. Further, the boundaries of the functionalbuilding blocks have been arbitrarily defined herein for the convenienceof the description. Alternative boundaries can be defined so long as thespecified functions and relationships thereof are appropriatelyperformed. Alternatives (including equivalents, extensions, variations,deviations, etc., of those described herein) will be apparent to personsskilled in the relevant art(s) based on the teachings contained herein.Such alternatives fall within the scope and spirit of the disclosedembodiments.

While examples and features of disclosed principles are describedherein, modifications, adaptations, and other implementations arepossible without departing from the spirit and scope of the disclosedembodiments. Also, the words “comprising,” “having,” “containing,” and“including,” and other similar forms are intended to be equivalent inmeaning and be open ended in that an item or items following any one ofthese words is not meant to be an exhaustive listing of such item oritems, or meant to be limited to only the listed item or items. It mustalso be noted that as used herein and in the appended claims, thesingular forms “a,” “an,” and “the” include plural references unless thecontext clearly dictates otherwise.

It will be appreciated that the present invention is not limited to theexact construction that has been described above and illustrated in theaccompanying drawings, and that various modifications and changes can bemade without departing from the scope thereof. It is intended that thescope of the invention should only be limited by the appended claims.

What is claimed is:
 1. A multispectral sensing device, comprising: anarray of pixel units, wherein: each of the pixel units comprises fourpixels in a two by two configuration; each of the pixels comprises aplurality of sub-pixels; and each of the pixel units includes at leastone pixel that includes at least two sub-pixels configured to detectlight of different wavelengths.
 2. The multispectral sensing device ofclaim 1, wherein the array is a Bayer-pattern array.
 3. Themultispectral sensing device of claim 1, wherein: each of the pixelscomprises four sub-pixels; each of the pixel units comprises sixteensub-pixels; and the sixteen sub-pixels are configured to detect light ofdifferent wavelengths or wavelength bands.
 4. The multispectral sensingdevice of claim 1, wherein: each of the sub-pixels comprises a μlens, anoptical filter, and a photodiode; the μlens is disposed above theoptical filter; the photodiode is disposed below the optical filter; theμlens is configured to direct impinging light to pass through theoptical filter; and the photodiode is configured to detect the lightpassed through the optical filter.
 5. The multispectral sensing deviceof claim 4, further comprising: a processing circuit coupled to theoptical filter and configured to tune a filtering range of the opticalfilter.
 6. The multispectral sensing device of claim 4, furthercomprising: a processing circuit coupled to the photodiode of each ofthe sub-pixels, wherein the processing circuit is configured to obtain atotal reading of all photodiode readings in each pixel; and output thetotal reading as the pixel's reading.
 7. The multispectral sensingdevice of claim 4, further comprising: a processing circuit coupled tothe photodiode of each of the sub-pixels, wherein the processing circuitis configured to obtain one photodiode's reading from all photodiodes ineach pixel; determine the pixel's reading based on the obtained reading;and output the determined reading as the pixel's reading.
 8. Themultispectral sensing device of claim 1, wherein, in each pixel: threeof the four sub-pixels are configured to detect visible light; and theother sub-pixel of the four sub-pixels is configured to detect infraredlight.
 9. The multispectral sensing device of claim 8, wherein: thethree sub-pixels are configured to detect red, blue, and green lightrespectively; and the other sub-pixel is configured to detect nearinfrared light.
 10. A multispectral sensing device, comprising: an arrayof pixels, wherein: each of the pixels comprises a plurality ofsub-pixels; and at least two of the sub-pixels in one pixel of the arrayare configured to detect light of different wavelengths.
 11. Themultispectral sensing device of claim 10, wherein the array is aBayer-pattern array.
 12. The multispectral sensing device of claim 11,wherein the plurality of sub-pixels are configured to detect light ofvarious wavelengths or wavelength bands different from each other andfrom sub-pixels of any of eight closest neighboring pixels.
 13. Themultispectral sensing device of claim 10, wherein: each of thesub-pixels comprises a μlens, an optical filter, and a photodiode; theμlens is disposed above the optical filter; the photodiode is disposedbelow the optical filter; the μlens is configured to direct impinginglight to pass through the optical filter; and the photodiode isconfigured to detect the light passed through the optical filter. 14.The multispectral sensing device of claim 13, further comprising: aprocessing circuit coupled to the optical filter and configured to tunea filtering range of the optical filter.
 15. The multispectral sensingdevice of claim 13, further comprising: a processing circuit coupled tothe photodiode of each of the sub-pixels, wherein the processing circuitis configured to obtain a total reading of all photodiode readings ineach pixel; and output the total readings as the pixel's reading. 16.The multispectral sensing device of claim 13, further comprising: aprocessing circuit coupled to the photodiode of each of the sub-pixels,wherein the processing circuit is configured to obtain one photodiode'sreading from all photodiodes in each pixel; determine the pixel'sreading based on the obtained reading; and output the determined readingas the pixel's reading.
 17. The multispectral sensing device of claim10, wherein, in each pixel: the plurality of sub-pixels comprise foursub-pixels; three of the four sub-pixels are configured to detectvisible light; and the other sub-pixel of the four sub-pixels isconfigured to detect infrared light.
 18. The multispectral sensingdevice of claim 17, wherein: the three sub-pixels are configured todetect red, blue, and green light respectively; and the other sub-pixelis configured to detect near infrared light.
 19. A multispectral sensingsystem, comprising: a Bayer-pattern array of pixels, each pixelcomprising a plurality of sub-pixels; and a processing circuit coupledto each of the sub-pixels and configured to tune a detection wavelengthband of the sub-pixel, causing the sub-pixels to be switchable betweendetecting light of the same and different wavelength bands.
 20. Themultispectral sensing system of claim 19, wherein: the pixels comprisesred, green, blue, and infrared pixels; each of the sub-pixels comprisesan optical filter; and the detection wavelength band of the sub-pixel istunable through tuning a filtering range of the optical filter.